\[ \definecolor{firebrick}{RGB}{178,34,34} \newcommand{\red}[1]{{\color{firebrick}{#1}}} \] \[ \definecolor{green}{RGB}{107,142,35} \newcommand{\green}[1]{{\color{green}{#1}}} \] \[ \definecolor{blue}{RGB}{0,0,205} \newcommand{\blue}[1]{{\color{blue}{#1}}} \] \[ \newcommand{\den}[1]{[\![#1]\!]} \] \[ \newcommand{\set}[1]{\{#1\}} \] \[ \newcommand{\tuple}[1]{\langle#1\rangle} \]

\[\newcommand{\States}{{T}}\] \[\newcommand{\state}{{t}}\] \[\newcommand{\Messgs}{{M}}\] \[\newcommand{\messg}{{m}}\]

roadmap

 

language processing

 

levels of analysis

 

rational speech act model

 

experimental data

 

model fits

 

reflection

processing?

incremental & predicitive processing

processing

  • comprehension of serially presented written or oral language input in context

 

incrementality

  • build a syntactic & semantic representations as the sentence comes in

    • how to deal with ambiguity? singular guess or parallel hypotheses?

 

predicitive

  • minimal sense: processing behavior is a function of current state

  • strong(est) sense: comprehender entertains hypotheses about the future

(Kuperberg & Jaeger 2016)

levels of analysis

levels of analysis

 

 

elephant

 

computational

 

algorithmic

 

implementational

(Marr 1982)

levels of analysis

LoA

(Franke & Jäger 2016)

algorithmic-level accounts

machine

mechanics of scalar implicature processing

what?

  1. compute SI
  2. verify SI
  3. integrate/cancel SI

 

when?

  1. scalar item
  2. full NP
  3. matrix clause

how?

  1. costly
  2. delayed
  3. cheap & immediate

 

whence?

  1. automatic
  2. voluntary
  3. context-dependent

method of choice: hypothesis testing

 

pick account

 

derive some (excentric) prediction

 

design experiment

 

refute & repeat

significance

(Platt 1964)

computational-level accounts

 

evolution

notions of "explanation"

 

 

description <———————————————–> reason

 

 

 

diagnostics

how would it work on Tralfamadore?

could you conceive of it without seeing any data?

trafalmadore

method of choice: models & comparison

data-generating models

modelGraph

statistical model comparison

justitia

expectation-based processing accounts

  • basic premiss: anticipation is an advantage
  • (maximally) predictive interpreter has lexico-syntactic expecations \(P(w_1, \dots, w_n \mid c)\)
    • operationalized by corpus frequencies of relevant structures
    • possible beam-search approximation
    • derived expectation about continuation \(P(w_{i+1}, \dots, w_n \mid w_1, \dots, w_i, c)\)

 

  • processing difficulty linked to distance between \(P(\cdot \mid w_1, \dots, w_\red{i}, c)\) and \(P(\cdot \mid w_1, \dots, w_\red{i+1}, c)\)
    • self-paced reading times (Smith & Levy 2013)
    • various ERP components, notably N400 amplitude (Frank et al. 2015)

(e.g., Jurafsky 1996, Hale 2006, Levy 2008)

rational pragmatic processors

rational speech act model

 

literal listener picks literal interpretation (uniformly at random):

\[ P_{LL}(t \mid m) \propto P(t \mid [\![m]\!]) \]

 

Gricean speaker approximates informativity-maximization:

\[ P_{S}(m \mid t) \propto \exp( \lambda P_{LL}(t \mid m)) \]

 

pragmatic listener uses Bayes' rule to infer likely world states:

\[ P_L(t \mid m ) \propto P(t) \cdot P_S(m \mid t) \]

 

interpretation as holistic: full & complete utterance

(e.g., Benz 2006, Frank & Goodman 2012)

incremental & predicitive interpretation

  • messages are word sequences: \(\messg = w_1, \dots, w_n\)

  • initial subsequence of \(\messg\): \(\messg_{\rightarrow i} = w_1, \dots w_i\)

  • all messages sharing initial subsequence: \(\Messgs(\messg_{\rightarrow i}) = \set{\messg' \in \Messgs \mid \messg'_{\rightarrow i} = \messg_{\rightarrow i}}\)

  • next-word expectation:

\[P_L(w_{i+1} \mid \messg_{\rightarrow i}) \propto \sum_{\state} P(\state) \ \sum_{\messg' \in \Messgs(\messg_{\rightarrow i}, w_{i+1})} P_S(\messg' \mid \state)\]

  • interpretation evidence:

\[P_L(\state \mid \messg_{\rightarrow i}) \propto P(\state) \ \sum_{\messg' \in \Messgs(\messg_{\rightarrow i})} P_S(\messg' \mid \state)\]

empirical measures

next-word

  • self-paced reading

  • eye-tracked reading

  • ERPs

  • …?

interpretation

  • visual worlds

  • mouse-tracking

  • …?

ERP traces of scalar implicature

some EEG studies on some

Noveck & Posada (2003)

  • ERPs on last word during reading: e.g. "Some people have fins/lungs/pets."
  • N400 amplitude: "pets" > "lungs"

Nieuwland et al. (2010)

  • similar to Noveck & Posada
  • two types of responders based on Autism Spectrum Quotient

Politzer-Ahles et al., (2013)

  • sentences following pictures
  • semantic vs. pragmatic violations
  • sust. negativity on underinformative quantifier
  • no N400 on aggregated ERPs at quantifier

Hunt et al. (2013)

  • sentences with pictorial contexts: controls for lexical associations
  • truth-value judgement after each sentence
  • semantic vs. pragmatic responders
  • pragmatic resonders' N400: false > underinformative > true & felicitous

 

Spychalska et al. (2016)

  • similar to Hunt, but more careful design
  • no explanatory role for ASQ scores
  • strong explanatory role for responder type
  • explicit focus on next-work prediction

experiment 1

participants & procedure

  • EEG recording of 25 native German speakers
  • picture (1500ms) -> sentence (500ms per word) -> truth-value judgement

sentence material

  • "Alle/Einige\(_1\) Punkte sind blau\(_2\), die im Kreis/Quadrat\(_3\) sind"
  • "All/some of the dots in the circle/square are blue/red"

visual stimuli

stimuli

computational-level rational processing

general assumptions

  • listener expects speaker to make a pragmatically felicitous utterance
  • listener does not give up on speaker rationality on the way (charity, forward induction, …)

 

experimental microcosmos assumption

  • possible meanings \(\States\): pairs of contexts (\(A\) - \(D\)) and speaker-intended shape
  • possible messages \(\Messgs\): "All/some dots are blue/red that are in the square/circle."

 

specific assumptions

  • listener knows context, but not shape
  • speaker chooses description for context and shape

next-word expectations vs. N400

  • incremental RSA predicts \(P_L(w_{i+1} \mid w_{1,\dots,i}, c)\)
  • correlating predicted next-word expectations grand-average early N400 (300-400ms):
    • \(r= 0.44\), \(p < 0.01\) in total
    • \(r = 0.81\), \(p < 0.001\) after exclusion of unexpected continuations

closer look: quantifier position

stimuli

stimuli

closer look: adjective position

stimuli

stimuli

closer look: shape noun position

stimuli

stimuli

semantic vs. pragmatic responders

semantic vs. pragmatic responders

repsonsefreqs

percentage of pragmatic responses per participant

predictions for response types

 

pragmatic responders

expect pragmatic speakers

\[P(\text{more informative true}) > P(\text{less informative true}) > P(\text{false})\]

 

semantic responders

expect literal speakers

\[P(\text{more informative true}) = P(\text{less informative true}) > P(\text{false})\]

(Franke 2012, Franke & Degen 2016)

results

  • correlation between predictions and observations (excluding redundant relative clauses):
    • \(r= 0.69\), \(p < 0.01\) for pragmatic responders
    • \(r = 0.57\), \(p < 0.01\) for semantic responders

stimuli

stimuli

further issues

experimental microcosmos

main issue

how to fix reasonable \(\States\) and \(\Messgs\)?

 

experimental microcosmos assumption

all (and only?) meanings and forms that occur in the experiment

 

prediction

massive influence of filler material

experiment 2

participants & procedure

  • EEG recording of 24 native German speakers
  • picture (1500ms) -> sentence (500ms per word) -> truth-value judgement

sentence material

  • "Einige\(_1\) Punkte sind blau\(_2\), die im Kreis/Quadrat\(_3\) sind"
  • "Einige\(_1\) Punkte sind blau\(_2\)"

visual stimuli

stimuli

results

behavioral data

only one participant consistently gave pragmatic judgements

ERP responses

no trace of pragmatic infelicity / expectations

conclusions

conclusions

  • incremental RSA seems feasible:
    • next-word expecations & accummulated interpretation evidence
    • genuine pragmatic expectations beyond / on top of lexico-pragmatic expectations

 

  • main challenges:
    • link functions for interesting experimental measures
    • how to fix \(\States\) and \(\Messgs\) in general?
    • how to scale up to more open-ended applications? tradeoff lexico-syntactic vs. pragmatic expectations?

appendix

quantifier

ERPs01

adjective

ERPs02

noun

ERPs03